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Abstract 



We show that inverse problems with a truncated quadratic regularization are 
NP-hard in general to solve, or even approximate up to an additive error. This 
, stands in contrast to the case corresponding to a finite-dimensional approxi- 

mation to the Mumford-Shah functional, where the operator involved is the 
identity and for which polynomial-time solutions are known. Consequently, we 
»^ ' confirm the infeasibility of any natural extension of the Mumford-Shah func- 

, tional to general inverse problems. A connection between truncated quadratic 

minimization and sparsity-constraincd minimization is also discussed. 

Keywords: inverse problems, Mumford-Shah functional, truncated quadratic 
regularization, sparse recovery, NP-hard, thresholding, SUBSET-SUM 

> 

in . 1 Introduction 

as _ 

■ Consider a discrete signal x G sampled from a piecewise smooth signal and 

, revealed through measurements y = Ax + e, where e S M'" is observation noise and 

f-*) ' A : I— >• M"* is a known linear operator identified with an m x N real matrix 

(representing, for instance, a blurring or partial obscuring of the data). Consider 
the truncated quadratic minimization problem, 

^ I X = argmin J'(x), 

N-1 

J{x) = \\Ax - y\\l + <3(^i+i - ^i)' (1) 
i=i 

with truncated quadratic penalty term Q{u) = a minjti^, /?} parametrized by a, /3 > 
0. Since its introduction in 1984 by Geman and Geman in the context of image 
restoration [SIEIIH], this problem has been the subject of considerable theoretical 
and practical interest, finding applications ranging from visual analysis to crack 
detection in fracture mechanics [9l[T0]. The choice of regularization is motivated 
as follows: Q desires to smooth small differences \xj^i — Xj\ < where it acts 
quadratically, but suspends smoothing over larger differences. 
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From a statistical point of view, the quadratic data- fidelity term \\Ax — y\\2 can 
be viewed as a log-likelihood of the data under the hypothesis that e is Gaussian 
random noise, while the truncated quadratic regularization term corresponds to the 
energy of a piecewise Gaussian Markov random field [UEHTJ. 

The truncated quadratic minimization problem is non-smooth and highly non-convex. 
However, several characterizations of the minimizers have been unveiled [3[H]. It is 
known for instance that minimizers exist and satisfy a "gap" property [5l[8]: the 
magnitude of successive differences of such solutions are either smaller than a first 
threshold or larger than a second, strictly larger threshold. These thresholds are 
independent of the observed data y = Ax + e and depend only on the regularization 
parameters a and /?. This dependence is explicit, so that a priori information about 
the thresholds can be incorporated into choice of regularization parameters. 

When A is the N x N identity matrix, the truncated quadratic objective func- 
tion can be viewed as a discretization of the Mumford-Shah functional^, which mo- 
tivated the variational approach for edge detection and image segmentation with 
its introduction in 1988. When A is the identity matrix as such, the truncated 
quadratic minimization problem can be solved in polynomial-time using dynamic 
programming [3]. However, for general m x N matrices A, existing algorithms for 
minimizing the functional ([1]) guarantee convergence to local minimizers at best [5] . 

In this paper, we show that the truncated quadratic minimization problem is NP- 
hard in general, certifying that the present convergence guarantees are the best one 
could hope for. Consequently, the Mumford-Shah functional ([2|) cannot be tractably 
extended to general inverse problems. 

2 Truncated quadratic minimization reformulated 

It will be helpful to recast the truncated quadratic minimization problem in terms 
of the discrete differences effectively decoupling the action of the 

regularization term Q. We may express this change of variables in matrix notation 

^In [4], the minimizers a; = if^r) of the TV-dimensional truncated quadratic minimization problem 
with A = I, parameters (a(]v), /3(jv)) = {N'^ct, N/S), and y^N) = y(j/N) identified with discrete 
samples from a continuous function y £ L°°[0, 1], were shown to converge to the mininiizer of the 
Mumford-Shah functional, 

X = argmin T{x), 
xeSBV[o,i] 

T{x) = / ({x-yf + a\\\/x\\l)dx + al3\S:,\, 

over the space SBV of bounded variation functions on [0, 1] with vanishing Cantor part. Note that 
SBV functions have a well-defined discontinuity set Sx of finite cardinality 1 5*2:1; see [S] for more 
details. 
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as n = Dx, with D : — )• ^ the discrete difference matrix, 

/ -1 1 \ 

^_ 0-11 ... 

V -11/ 

The nuh space of £), which we denote in the fohowing by J\f{'D), is simply the 
one-dimensional subspace of constant vectors in M^. The orthogonal projection of 
a vector x G onto this subspace is the constant vector c whose entries coin- 
cide with the mean value SjLi of while its projection onto the orthogonal 
complement of J\f{T>) is given by the least squares solution D^Dx, where is the 
pseudo-inverse matrix of D in the Moore-Penrose sense. These observations yield 
the orthogonal decomposition x = D^Dx + c, or, incorporating the substitution 
u = Dx, the decomposition x = D^u + c. 



Minimization of J, recast in terms of the variables u and c, becomes 



(c, n)= argmin J{c,u) 

ceN{D), -uGM^-i 



J{c,u)= AD^u + Ac-y +^Qii 

^ i=i 



■J)-, 



(2) 



where the primal minimizer x and (c, u)-minimizer (c, u) are interchangeable accord- 
ing to X = D^u + c. 



If the null space of A contains the constant vectors, such as if A = TD for an 
m X [N — 1) matrix T, the minimization problem ([2]) reduces to a function of u only, 



u = argmm 



2 ^"^ 

ad\ 



'u-y 



2 

i=i 



Making the substitution A = TD and using that DD^ = L we see in particular that 
any optimization problem of the form d = argmin^gj^jv-i \\Tu — y\\2 + ^f=i^ Q{uj) 
can be identified with an instance of a truncated quadratic minimization problem ([T|). 
To summarize. 

Lemma 1. Let T : i— )• be a linear operator identified with a matrix of 
]^m.x(Af-i)_ ji^g minimization problem 

N-l 

u = argmin \\Tu — y\\^ + Q{uj) (3) 

corresponds to a truncated quadratic minimization problem 

N-l 

X = argmin \\TDx — y\\\ + Q{xjj^i — Xj) (4) 



X 



in the sense that x = D^u is a minimizer for if u minimizes ([3]), while u = Dx 
is a minimizer for 1^ if x is a minimizer for . 
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3 Reduction to SUBSET-SUM 



Recall that the complexity class NP consists of all problems whose solution can be 
verified in polynomial time given a certificate for the answer. For example, the prob- 
lem SUBSET-SUM is to determine, given nonzero integers ai, . . . , and C, whether 
or not there exists a subset S" of {1, . . . , A;} such that X^jg^ = C. This problem is 
in NP because given any particular subset S, we can easily check whether or not its 
corresponding sum is zero. 

Further recall that a polynomial-time many-one reduction from a problem A to 
a problem B is an algorithm that transforms an instance of A to an instance of B 
with the same answer in time polynomial with respect to the number of bits used to 
represent the instance of A. Intuitively, this captures the notion that A is no harder 
than B, up to polynomial factors, and accordingly one may write A < B. Finally, a 
problem B is called NP-hard if every problem in NP is reducible to B. (Note that 
A < B and B < C imply j4 < C, so if an NP-hard problem B reduces to a problem 
C, then C is NP-hard as well.) NP-hard problems can not be solved in polynomial 
time unless P=NP. 



In order to prove our NP-hardness result, we show that the known NP-hard prob- 
lem SUBSET-SUM admits a polynomial-time reduction to an instance of the trun- 
cated quadratic minimization problem. Moreover, we show that any algorithm that 
could efficiently approximate this minimum (to within an additive error) could solve 
SUBSET-SUM efficiently as well; that is, the search for even an approximate solution 
to a truncated quadratic minimization problem is NP-hard as well. 

Theorem 2. Let oi, . . . ,0^ and C be given nonzero integers. Then there exists a 
subsets of{l,...,k} such that J2ieS^i~^ if and onlyifmm^^^2kf{x) < k, where 

k 

f{x) = I C - ^ J QiXi ) -\-P -^{l-Xi- Xi+kf 

1=1 




+ ^min(^l,-|J , 



with < e < ^^^'';|^,|^ and P > || . Moreover, this minimum is never strictly between 
k and k -\- ^. 

Proof. Call a subset 5 C {1, . . . , fc} good if J2ies = C. If a good subset S exists, 
we may set 

fl ifiG5, , fo ifieS, 

Xi = { and Xj+k = < 

\0 ifi^5, ^ \l ifi^S, 

for \ < i < k. Then f{x) = k because the first and second terms vanish, and there 
are exactly k nonzero XjS. Therefore, if a good subset exists, the minimum is at 
most k. 
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Suppose no good subset exists, yet there exists x such that f{x) < k + \. Con- 
sider the k pairs of coordinates Xj,Xj-|_fc for 1 ^ i ^ /c. If both and 
were less than e, a single summand in the second term already exceeds 2k, as 
P ■ {1 — Xi — Xj_|_fc)^ > P • (1 — 2e)^ > 2k. Therefore, at least one of |xj| and 
exceeds e and the third term is already at least k. If more than one of and 
exceeded e, then the third term would be at least fc + 1, so exactly one of the 
coordinates in each pair exceeds E in ctbsolute value. If \xi\ ^ 6, then [x^-i-fc — 1| < 2e 
as otherwise P • (1 — — Xj+fc)^ > P ■ {e)'^ > 2k; the symmetric holds if |xj+fc| < e, 
so all of the Xi Eire within 2e of either or 1. 

Let Xi be the closer of and 1 to Xj. Then because no good subset exists and 



C and the Oj are integers. 



c - Ell 



diXi 



> 1. It follows that the first term 



k \ ^ 

C — Ylii=i o-i^i) ^ (1 ~ 2eai)'^ > (i)^ = i. But the third term was already at 
least k, so this is a contradiction. Therefore, if no good subset exists, we must have 
miua; /(x) >k + \. □ 

Corollary 3. Solving the truncated quadratic regularization problem, even to within 
an additive error, is NP-hard. 

Proof. In light of Lemma [H minimization of the function / is a truncated quadratic 
minimization problem, with m = k + 1 and N = 2k. Therefore, we have reduced 
the known NP-hard problem SUBSET-SUM to a truncated quadratic minimization 
problem. It remains to verify that this reduction is polynomial-time. To see this, 
note that Theorem [2] ensures that the minimum of / is either at most k or at least 
k + thus, we only need to approximate each of the polynomially-many entries in 
the matrices and vectors in / to within a number of bits that is polynomial compared 
to the number of bits needed to represent J2i I'^il- ^ 



4 Connection to sparse recovery 

The only properties of the quadratic regularization term min{l,x?/e^} needed for 
Theorem [2] were that it be bounded between and 1, equal to if Xj = 0, and equal 
to 1 if |xj| > e. Indeed, Theorem [2] holds for any regularization term satisfying these 
properties; for example, one could consider hard thresholding, 

Jo X = 0, 

which generates the £o "counting norm" ||x||q = J2f=i l^jlo- then reprove the 
known result that the ^g-regularized optimization problem, 

u = argmin \\Tu — + 7 \\u\\q , 

is NP-hard in general. This functional is of considerable interest in the emerging 
area of sparse recovery, as it is guaranteed toproduce sparse solutions for sufficiently 
large 7 and over a certain class of matrices q In this light, the truncated quadratic 

^See [3] for matrix constructions that admit polynomial-time recovery algorithms for the £0- 
regularized optimization problem. All constructions at present involve an element of randomness, 



5 



minimization problem may be interpreted as a relaxation of the £o-regularized opti- 
mization problem, and our main result as showing that even such relaxations of the 
•£o-regularized functional are NP-hard. 
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and a complete characterization of such matrices forms the core of the area known as compressed 
sensing. 
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